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Abstract 

Facing a customer market with rising demands for cloud service dependability and security, trustworthiness evaluation 
techniques are becoming essential to cloud service selection. But these methods are out of the reach to most customers as 
they require considerable expertise. Additionally, since the cloud service evaluation is often a costly and time-consuming 
process, it is not practical to measure trustworthy attributes of all candidates for each customer. Many existing models 
cannot easily deal with cloud services which have very few historical records. In this paper, we propose a novel service 
selection approach in which the missing value prediction and the multi-attribute trustworthiness evaluation are commonly 
taken into account. By simply collecting limited historical records, the current approach is able to support the personalized 
trustworthy service selection. The experimental results also show that our approach performs much better than other 
competing ones with respect to the customer preference and expectation in trustworthiness assessment. 
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Introduction 

Cloud computing has become the driver for innovation in the 
recent years, from startups (e.g. Dropbox, Instagram) to 
established enterprises (Samsung). They are all using cloud 
computing to better serve their customers around the world [1]. 
Cloud service is also gaining wide acceptance and becoming 
popular to individuals as it reduces hardware and licensing costs, 
and it is scalable and allows users to work from any computer 
anywhere. 

Several leading IT enterprises including Google, IBM, Micro- 
soft, and Amazon have started to offer cloud services to their 
customers [2-4] . While many small and medium-sized enterprises 
(SMEs) and individual customers prefer to apply cloud services to 
build their business system or personal applications, they are often 
facing two major challenges at the selection time: (1) multiple 
cloud services are often available by different venders providing 
similar functional properties (i.e., "functionally-equivalent"). Cus- 
tomers usually lack appropriate, qualified, sufficient information 
and benchmarks to assess cloud services with regard to individual 
preferences and market dynamics [5]; (2) although cloud service 
vendors are struggling to improve service quality and perfor- 
mance, cloud computing are not necessarily trustworthy - 
unhandled exceptions and crashes may cause cloud service to 



deviate dramatically from the expectation [6,7]. Therefore, there 
is an increasing demand to help the non-expert customers with the 
selection of trustworthy cloud service. 

The trustworthiness of cloud service affects customers' percep- 
tion towards service quality, which has significant bearing on 
customer satisfaction and royalty. The trustworthy attributes 
include reliability, scalability, availability, safety, security, etc [8- 
1 0] . Designing a general and comprehensive analytical model for 
trustworthiness evaluation is challenging, as the model needs the 
assessor to achieve, in reasonable time, useful results to determine 
the best service option. Due to their commercial value (similar to 
online recommendation system), several evaluation models [1 1— 
14] have been proposed by academia and industry lately. These 
models focus on quantitative analysis and evaluate trustworthiness 
through a collectively exhaustive dataset. 

Except for some specific cases, the assessment dataset remains 
very sparse due to the costly and time-consuming nature of cloud 
service invocation. Intuitively, without sufficient data, fair review 
of cloud services cannot be achieved by existing evaluation 
methods [9,15]. Fortunately, cloud vendors can collect historical 
records (QpS values, customer ratings, etc) from different cloud 
applications in cloud computing environment. With the vast 
amount of collaborative filtering (CF) technologies available in the 
field of online recommendation system, we believe there is a strong 
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theoretical foundation to derive a generic trustworthiness model to 
support the evaluation of cloud service. 

There have been some attempts to improve the accuracy of 
cloud service assessment by a CF process. However, very little 
attention is paid to the trustworthiness of cloud service, and no 
interest is given to the case when significant attribute values are 
missing. The lack of general and formal methodology can be 
attributed to the large process gap between the cloud service 
recommenders and trustworthiness researchers. To deal with this 
challenge, we propose a new CF approach to make use of hidden 
information (i.e. experience usability, value distribution) to 
measure the similarity between different services. Moreover, to 
support personalized selection of cloud services, we also provide a 
natural treatment for multi-attribute aggregation taking into 
account customer's preference and expectation. 

Background 

In the current market, multiple cloud services of similar 
functions are often available for specific domains. For example, 
in cloud storage service (e.g. data service, online file system, online 
backup plan), over 100 functionally-equivalent cloud services are 
offered by vendors. Some typical examples can be found in 
Table 1 . Given the lack of cloud computing experience of non- 
expert customers, it is tedious to manually select an appropriate 
candidate from a set of functionally-equivalent services. Therefore, 
cloud service evaluation through quality analysis has gained much 
attraction among service-oriented computing and cloud comput- 
ing communities over the past two decades. 

Given the intricate interactions among QpS (Quality of Service) 
attributes, customer preferences and market dynamics that jointly 
influence the perceived quality of cloud services, developing a 
market-relevant analytical model is crucial to cloud service 
selection [16—18]. Due to their commercial value and the 



associated research challenges, many researchers and practitioners 
have studied the topics. Two types of service selection models are 
widely examined: evaluation-focused service selection models and 
prediction-focused service selection models. 

By achieving market-relevant evaluations, customers can 
identify risks and benefits of each cloud service application and 
choose the best for adoption. The most employed evaluation 
models include: AHP-based cloud service ranking [19], reputa- 
tion-aware service rating [20], trust-aware service selection [21], 
brokerage-based selection [22], SLA-based cloud trustworthiness 
estimation [11], trustworthy service selection [23]. Although these 
techniques can accurately and exhaustively estimate service 
quality, their implementation is time-consuming and costly. 

Instead of real-world cloud service invocations, the prediction- 
focused service selection models can produce QpS values or 
service ranking using collaborative filtering (CF). The CF 
approaches for cloud service selection can be categorized as: 
item-based approaches [24], customer-based approaches [25], 
their fusion approaches [26], model-based approaches [27], and 
ranking-oriented approaches [28], where the first three categories 
are rating-oriented approaches. These approaches help assessors 
predict the missing attribute values by exploiting neighbors' usage 
experiences. Several collaborative filtering approaches for cloud 
service selection have been studied, but they did not consider 
customer preference and expectation in trustworthiness assess- 
ment. 

In the prediction process, similar neighbors (customers or 
services) are identified to generate useful collaborative informa- 
tion. Popular choices for similarity estimation include Pearson 
correlation coefficient (PCC) [29] and vector similarity (VS) [30]. 
Since these measures only consider the numerical relationship 
between different ratings, they remain imprecise and confusing for 
estimating the neighbor similarity to support missing value 



Table 1. 


Online cloud storage services. 








Vender 


Cloud Service 


Feature 


Pricing 


Amazon 


EBS 


Storage Service 


$0.1 per GB-month, $0.1 per 1 million I/O requests 


Amazon 


S3 Standard 


Storage Service 


$0,095 per GB-month, $0,005 per 1000 requests 


Google 


Google Cloud Storage 


Storage Service 


$0,085 per GB-month, $0.01 per 1000 ops-month 


IBM 


SoftLayer Object Storage 


Storage Service 


$0.1 per GB-month 


Microsoft 


Azure Data Service 


Storage Service 


$0,095 per GB-month, $0.01 per 100000 I/O requests 


Apple 


iCIoud 


Storage Service 


$20 for 10 GB upgrade 


GoGrid 


GoGrid Cloud Storage 


Storage Service 


$0.12 per GB-month 


JustCloud 


JustCloud Cloud Storage 


Storage Service 


$3.95 per month, unlimited storage 


ZipCloud 


ZipCloud Online Storage 


Storage Service 


$6.95 per month, unlimited storage 


AT&T 


Synaptic Storage 


Storage Service 


Unknown 


LiveDrive 


Livedrive Backup Plan 


Backup System 


$6 per month, 2 TB storage space 


CrashPlan 


CrashPlan Backup Plan 


Backup System 


$5.99 per month, unlimited storage 


Carbonite 


Cloud Backup Services 


Backup System 


$59.99 per year, unlimited storage 


FlexiScale 


FlexiScale Public Cloud 


Platform Service 


$17 per 1000 unit-hour 


AppNexus 


AppNexus Cloud 


Platform Service 


Unknown 


Rackspace 


Mosso cloud files 


File System 


$0.75 per GB-month 


HighTail 


HighTail 


File System 


$15.99 per month, unlimited storage 


Amazon 


SimpleDB 


Database 


$0.12 per GB-month 


doi:1 0.1 371/journal.pone.0097762.t001 
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prediction. Concerned that PCC may overestimate the similarities 
of negative services, Zheng et al. [26] propose a significance weight 
and modify PCC to improve the accuracy of similarity compu- 
tation in service recommendation. However, the significance 
weight affects the similarity computation of positive services with 
more usage experiences. To address this problem, Ding et al. [3 1] 
define a convex function (usage structure factor) to reflect the 
usability of customer experience. 

While a great number of researchers have focused on the trust- 
aware service selection and recommendation, littie attention has 
been devoted to the role of customer preference and expectation in 
multi-attribute trustworthiness evaluation [32]. In addition, large 
quantities of works offer some valuable clues to discern between 
different services, the significances arising from value distribution 
is seldom considered. Thus, we will here combine evaluation- 
focused and prediction-focused approaches to propose a novel 
trustworthiness evaluation method which will fully utilize the 
information of similar services and customer's experience, and 
take into account both the missing attribute value prediction and 
the multi-attribute trustworthiness evaluation at the same time. 

Methods and Materials 

Based on the fact that the size and rate of growth in customers 
outweigh the expansion of delivered services in the cloud 
computing market, we employ item-based CF approach rather 
than the user-based or their fusion approach to produce the 
missing attribute values in trustworthiness evaluation. Motivated 
by the observation that experience usability and value distribution 
could provide valuable insight and distinctive information in the 
CF process, we create a new similarity measure for enhancing the 
prediction performance. 

Pearson Correlation Coefficient 

To make an accurate prediction, we first estimate the similarity 
between different cloud services. Given a service selection problem 
consisting of M customers and jV services, the customer-service 
matrix for missing value prediction is denoted as 



91,1 



qu, i 



qi,N 



qM,N 



(1) 



where the entry q m „ denotes a historical record (QpS value or 
customer rating) of cloud service cs n made by customer u m , 
"<lm,n = nulF' states that u m didn't invoke cs n yet. 

Pearson Correlation Coefficient (PCC). [29] Taking use of 
numerical distance to estimate the correlation between different 
services, PCC has been successfully adopted for recommendation 
system evaluations. Let cs„ and cs v be two services, U v be the 
subset of customers who have invoked both cs„ and cs m then PCC is 
applied to calculate the similarity between cs n and cs v by 



Sim(cs n ,cSy) ■■ 

E 



/net///, 



(qm,n-q„)(qm,y-q y ) 



(2) 



where Sim(cs m cs„) is in the interval of [— 1, 1], q n and q y stand for 
the average values of cs n and cs„ made by different customers. 
However, as noted in Ref. [26], PCC always overestimate the 



similarities of negative services, which are actually not similar but 
happen to have similar usage experience made by few customers. 
Table 2 shows a simple customer-service matrix which contains six 
customers (ui to a 6 ) and ten cloud services (cs\ to cs w ). When 
utilizing Eq. (2), we calculate the PCC values between the services, 
and get the following relation: Sim(csi, cs 3 )>Sm{csi, a 4 )>S , m(a 1 , 
cs 2 ), which indicates cs 3 is more similar to cs\ than cs 2 and cs^. It is 
clearly contrary to the reality due to the limited usage experience. 
Therefore, it is necessary to reinforce the similarity information in 
the CF process. 

Significance estimation 

It seems logical to believe that some cloud services in customer- 
service matrix may have high significances in making recommen- 
dations [33,34]. For instance, a cloud service, which has more 
useful historical records, may be regarded as more important 
compared with a negative service. PCC is only related to the 
numerical distance between different services, but it has nothing to 
do with the statistical features of historical records. For this case, 
we introduce two types of significances arising from the experience 
usability and value distribution of historical records, respectively. 

Estimating the experience usability. To determine the 
significance of neighbors in a CF process, one often assumes a 
linear relationship between usage experiences and neighbor 
significances [26,31]. One difference of our work from traditional 
CF approaches is that we apply a distance measurement method 
to estimate the experience usability in customer-service matrix. 
During the distance measurement, Jaccard's coefficient [35] is 
frequently employed to estimate the discrimination of asymmetric 
information on binary variables. Before integrating Jaccard's 
coefficient into our similarity measure, we map the original 
customer-service matrix into a rectangular binary matrix as 
follows: 



Am,\ 



9i,n 



9M.JV. 



mqmj, =null)THEN(b mJI =0) 



ELSE(bm,„ = l) 



VM,N _ 



(3) 



where the entry b m ^„ — 1 denotes the customer u m has invoked the 
service cs„ previously, whereas b m-n = 0 denotes that u m didn't 
invoke cs n . Let \U„\ be the number of customers who has invoked 
cs n before, and | U nj | be the number of customers who invoked 
both cs„ and cs v . We use the Jaccard's coefficient J v to reflect the 
rise of significance due to the experience usability, which can be 
expressed mathematically as: 



\U„,y 



\U„\ + \U y \-\U n J JL 



(4) 



( V« V b m,y) 



where J nj is in the interval of [0, 1] , and a larger J v value indicates 
that the historical records made over cs v is more useful in the CF 
process. J„j,=J J , ! „ holds for all services, which is consistent with the 
intuition that the similarity between cs v and cs n is only related to the 
subset of historical records made by the customers who have 
invoked both cs n and cs v . 

Based on the customer-service matrix in Table 2, we get the 
significances arising from the experience usability for each service, 
as shown in Table 3. The values shown in grey are calculated for 
the negative service cs%. As observed from Table 2, cs$ has only 
been invoked twice. Consequendy, his experience usability values 
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Table 2. A simple customer-service matrix. 





«1 


"2 


"3 


cs 4 


«5 


«6 


CSj 


«8 


«9 


«10 




0.9 


0.7 


null 


1 


0.8 


0.6 


0.4 


0.7 


nu// 


null 


"2 


0.8 


0.7 


0.8 


0.4 


0.6 


0.6 


0.7 


null 


0.9 


0.8 


"3 


0.9 


0.8 


null 


0.6 


0.5 


0.5 


0.5 


null 


nu// 


1 


"4 


0.8 


0.9 


null 


1 


null 


0.8 


0.6 


0.7 


0.6 


nu// 


"5 


0.7 


0.6 


null 


0.5 


0.7 


null 


0.8 


null 


0.4 


0.9 


"6 


null 


0.8 


0.9 


0.6 


null 


null 


0.9 


0.8 


0.8 


0.7 
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are significantly lower than other services (e-g-J\;>,«J\,2)- We can 
infer that integrating J njI into similarity measure will notably 
reduce the influence of negative service. It is worth noting that 
neither PCC nor J v can distinguish between cs 2 and CS4, since they 
do not have sufficient power to detect the crucial difference in 
value distributions. 

Estimating the value distribution. The neighbors which 
have the same PCC similarity may have different value 
distributions. It is necessary to detect more hidden information 
in the customer-service matrix for significance estimation. For this 
case, we propose a method to discriminate neighbors' significances 
arising from their unique value distributions. In practice, the 
customer-service matrix is very sparse due to limited usage 
experiences. Therefore, we will ignore the historical records made 
by the customer u, where u ^ U nj ,. Let Z) n = {</„, „ | u, 



e U njl } and 



{imj I u m e U njI } be the historical records in similarity 
computation made over cs n and cs m and | D y | be the cardinality of 
D„, and dom(D„) be the domain of D n subject to the following 
constraints: 



dom{D n )^>[q n ,q+ 



q n =MIN{q m<n \u m eU nt y), 
q+ = MAX [q„ hn \u m e U n ,y) ■ 



Following dom(D„), the dataset D y can be grouped into three 
categories: 



Dp" = {q m ,y\u m eU„,y,q m , y e(- co,q~)}, 
Dy" = {q„ KV \u m eU„ t) „q„ v ,e[q- ,q+]}, 
D y~" = {qm,y\UmeU n y,q m ,ye(q+ , + oo)}. 



(6) 



Since D y is a finite discrete dataset, the probability of each category 
can be computed as: 



Pl- 



\D<"\ 2 

— - — , PZ - 

\Dy\ ,P > 



\d;»\ 3 



\D> n \ 

\Dy\ 



(7) 



where ^J, . [/>',' 



rt-l { P 'y) =h and \D^"\ + \D-"\ + \D>-' l \ = \D y \. From 
the information entropy aspect, we use the following expression to 
detect the difference between the value distributions of cs n and csy. 



VD ny = 1 ■ 



(8) 



where 



^log 2 (p / ')) denotes the information 



entropy of D y , and H max denotes the maximal entropy in customer- 
service matrix, respectively. VD V is a linear function defined in [0, 
1]. From the maximum entropy principle [36], we have 
fmai = log2(3). Thus, Eq.(8) can be rewritten as 



VD„, y = \- 



-Py 10g 2 ^) 



log 2 (3) 



(9) 



where VD njl attains its unique global minimum VD^' N = 0 if 
p =PZ = Py = l/3> otherwise it attains global maximum 
VD^ AX = \ when 3»*=1. 

We can thus calculate the significances VD n ^ arising from the 
value distribution using Eq.(9) over the customer-service matrix in 



Table 3. Significances arising from the experience usability. 





«1 


«2 


"3 


«4 


«5 


«6 


«7 




«2 


0.833 
















«3 


0.167 


0.333 














CS t 


0.833 


1 


0.333 












"3 


0.8 


0.667 


0.2 


0.667 










"6 


0.8 


0.667 


0.2 


0.667 


0.6 








«7 


0.833 


1 


0.333 


1 


0.667 


0.667 






ess 


0.333 


0.5 


0.25 


0.5 


0.167 


0.4 


0.5 






0.5 


0.667 


0.5 


0.667 


0.333 


0.333 


0.667 


0.4 




0.5 


0.667 


0.5 


0.667 


0.6 


0.333 


0.667 


0.167 0.6 
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Table 4. Significances arising from the value distribution. 





«1 


"2 


«3 


cs 4 cs 5 


«6 


cs, 


«8 


cs 9 








1 


1 


1 0.488 


0.369 


0.387 


1 


1 


0.421 


«2 


0.545 




0.685 


1 1 


0.488 


1 


0.421 


1 


0.488 


«3 


1 


0.369 




1 1 


1 


1 


1 


1 


0.369 


CS A 


0.387 


0 


1 


0.054 


0.054 


0.079 


0.421 


0.488 


1 


"3 


0.369 


0.488 


1 




0.421 


1 


1 


1 


1 


«6 


0.488 


0.488 


1 


1 1 




0.488 


1 


1 


1 


«7 


0.387 


0.421 


0.369 


1 0.488 


0.488 




1 


1 


0.369 


csa 1 1 1 1 1 1 1 11 


«9 


0.421 


0.488 


1 


1 1 


0.369 


0.421 


0.369 




0.421 


«10 


0.421 


0.421 


1 


1 1 


1 


1 


1 


0.685 
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distribution to determine the similarity between different services. 
With the definition of similarity measure defined in Eq.(ll), for 
every cloud service in customer-service matrix, we rank their 
neighbors and select the top-k most similar services to make 
missing value prediction. Following the top-A similar service 
defined in [26], we get 

CS k n ={cs r \cs r eCS„, Sim s (cs„,cs y )>0,y^n}, (12) 

where CS„ denotes the neighbor set of cs„ in customer-service 
matrix, and Sim (cs n , «„) denotes the similarity between cs„ and cs v . 
For the customer-service matrix in Table 2, we set a to 0.8 to 
obtain the similarity measures between different services (see 
Table 5). The top 3 neighbors of each service are marked in grey 
areas as seen in each column. 

Missing value prediction. With the exponential growth of 
cloud service on the Internet, service recommendation techniques 
like QpS-aware CF approaches have become increasingly 
important and popular [37]. Based on our similarity measure, 
we propose an enhanced item-based CF approach (named as JV- 
PCC) to reinforce the prediction performance. To predict the 
missing value q m>n of service cs„ for customer u m , we first determine 
the objective weight of each similar neighbor: 



Table 5. Similarities between different services. 







«1 




"3 


cs 4 










cs 9 








0.361 


0.334 


0.366 


-0.099 


-0.421 


-0.663 


-0.24 


0.402 


0.141 




0.323 




0.403 


0.484 


-0.379 


0.376 


-0.153 


0 


0.213 


-0.081 


«3 


0.334 


0.34 




0.223 


0.36 


0.36 


0.259 


0.4 


-0.165 


-0.212 


CS t 


0.314 


0.387 


0.223 




0.336 


0.303 


-0.494 


-0.347 


-0.166 


0.046 


"5 


-0.096 


-0.326 


0.36 


0.453 




0.324 


-0.099 


-0.334 


-0.464 


-0.432 


«6 


-0.435 


0.376 


0.36 


0.408 


0.39 




0.145 


-0.312 


-0.208 


-0.405 


«7 


-0.663 


-0.135 


0.189 


-0.605 


-0.085 


0.146 




0.549 


0.032 


-0.451 


«8 


-0.24 


0 


0.4 


-0.43 


-0.334 


-0.312 


0.549 




0.518 


-0.334 


«9 


0.324 


0.184 


-0.165 


-0.193 


-0.464 


-0.152 


0.027 


0.393 




-0.395 


«10 


0.14 


-0.079 


-0.268 


0.04 


-0.432 


-0.405 


-0.544 


-0.334 


-0.432 
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Table 2. Table 4 shows the values of these significances. The 
values shown in grey are calculated for the cloud services cs 2 and 
CS4. 

Similarity measurement adopting significance. After we 
have defined the two types of significance for each service, we can 
then create the similarity measure, Sim{cs m cs s ), which takes into 
account the significance previously defined. To estimate the 
significance as accurately as possible, we identify the significance of 
cs v with respect to cs n as a linear combination of J njl and VD njl , such 
that: 

SIG n y =ax/ w +(l-«)xKD v , (10) 

where a is defined to determine how much our significance relies 
on experience usability and value distribution. If a = 0, we only 
extract the experience usability for conducting significance 
estimation, and if a = 1 , we consider only the value distribution. 
Hence, the similarity measure can be written in standard form: 

Sim s (cs n ,cs y ) = SIG„j, x Sim{cs n ,cs y ) (11) 

where SIG^ denotes the significance of cs„ with respect to cs m and 
Sim(cs„, cs B ) denotes the PCC value between cs„ and cs v . Different 
from existing similarity measures, our approach employs not only 
numerical distance but also usage experience as well as value 
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Table 6. Predicted attribute values. 

































0.843 












0.609 


1 


"2 
















0.808 






"3 






0.865 










0.74 


0.842 




"4 






0.85 




0.8 










0.887 


"5 






0.831 






0.593 




0.76 






"6 


0.865 








0.703 


0.749 
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attributes are often divided into quantitative and qualitative 
attributes, of which the former are objective measures (e.g. QpS 
value), and the latter are subjective customer ratings. In addition, 
quantitative trustworthy attributes can be grouped into two classes: 
"benefit" and "cost". For "benefit" ("cost") attribute, e.g. 
throughput [response-time], the higher (lower) its value is, the greater 
the possibility that a customer would choose it becomes. In our 
model, qualitative attributes are also considered as "benefit" 
attributes. Let q m „ be the attribute value of cs„, and the attribute 
utility (risk-neutral) H m n has the following form: 



' „ Sim s (cs„,cs„) 

where cs„ eCS^ denotes a similar neighbor of es m while Sim \es,„ cj„) 
denotes the similarity between cs n and cs v . The objective weights 
define the relative importance of each similar neighbor in the CF 
process. Next, we attain a prediction by a classic aggregation 
function: 



qm,y ¥"a4l 



(14) 



where q mj , denotes the historical record of cs y made by customer u m . 
In practice, local runtime environment (e.g. network bandwidth) 
and customer's rating style may significantly influence the 
historical records over delivered services. However, the above 
function deems inappropriate as it is calculated through only one 
customer. To address this problem, JV-PCC predicts the missing 
attribute value by the following equation: 



= <7,!+(<7« + - 



1n 



E 



qm,y+null 



In 



-<7y 



<?/-</; 



(15) 



where w nj , denotes the objective weight of cs y with respect to cs m 
while q~ and denote the minimum and maximum values of 
service cs y , respectively. Table 6 displays the values estimated for 
the missing records in Table 2. In the experimental examples, both 
customer-based and service-based neighborhood information were 
adopted for approximating the missing value. 

Trustworthiness-aware service selection 

Several models, focusing on the quantitative measurement of 
service trustworthiness, have been proposed in Refs. [9,18,38]. 
However, different customers have different preference and 
expectation in service selection. A diorough understanding into 
these factors is essential to ensure effective evaluation finding. 
Here, we introduce a cloud service evaluation model, which helps 
aggregate trustworthy attributes by considering customer's prefer- 
ence and expectation. 

Attribute utility determination. To make use of observed 
or estimated values, we need to know that different attributes may 
have inconsistent dimensions. The results in [32] show that utility 
can be used to identify an entity's trustworthiness. Therefore, we 
first derive the utility from the customer-service matrix so as to 
ensure their values are in the range of [0, 1]. Trustworthy 



C[m,n q n 



, qm.ns" benefit" , 



(16) 



q„,. n e cost 



where q m and q^ denote the minimum and maximum attribute 
values for customer u m , and they are subject to the following 
constraints: 



q~ =MIN(q m ,n\n=\,...,N), 
q+=MAX(q m , n \n = l,...,N). 



(17) 



The attribute utility H m „ is in the range of [0, 1], where a larger 

is more satisfied with the service 

Customer satisfaction estimation. From influential theory 
in marketing science, we consider that the perception of cloud 
service trustworthiness is a customer satisfaction function, which 
includes customer preference and expectation attributes. In 
general, customer satisfaction function should exhibit two charac- 
teristics: (1) given the same expectation, a trustworthy cloud service 
is weighed much more heavily than an untrustworthy service. This 
effect is reflected in the derivation of attribute utility; (2) customer 
satisfaction slighdy increases when attribute utility surpasses a 
certain value (expectation), and significandy decreases when 
attribute utility falls below expectation [39]. We formalize this 
interaction as a piecewise linear function: 



H > H ex P 



H„,JH nun -H^ + If, H m , n <H^, 



(18) 



where C m> „ is constrained to 0<C m „^l; the parameter 8 regulates 
the impact of customer preference on perceived trustworthiness; 
and i/ :xp denotes the customer expectation with regard to selecting 
trustworthy cloud service. As shown in Fig. 1, C m _„ is continuous 
(i.e. the piecewise function converges at H m n = //* xp ). 
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H 
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Figure 1. Customer satisfaction function C mn . (a) and (b) depict 
the distributions of customer satisfaction as recorded at the fixed 
expectation H exp = 0.7 and H exp = 0.9, where the parameter 6 is varied 
from 2 to 6 in increment of 2. It can be observed that the rate of change 
in customer satisfaction differs significantly when H mn falls below and 
exceeds the expectation. 
doi:1 0.1 371 /journal.pone.0097762.g001 

The severity and rate of satisfaction (controlled by lf xp and S, 
respectively) reflect different customer's tolerance to untrustworthy 
candidates. Let fl exp = 0.7, and S = 2. Table 7 shows the customer 
satisfaction for each attribute value (historical record or predicted 
value), which corresponds to the original customer-service matrix 
in Table 2 and the predicted missing values given in Table 6. 

Trustworthy attribute aggregation. After estimating cus- 
tomer satisfaction and ensuring the value of C m n in the interval of 
[0, 1], the degree of trustworthiness (alias "trust value" [40]) of 
each cloud service in customer-service matrix can be achieved by 
aggregating trustworthy attributes. Let C l mn ..C J mn be the customer 



Table 7. Customer satisfaction. 



satisfaction on a set of specified attributes A\..Aj, then the trust 
value of cs„ is computed as: 



trust„ - 



Ey=i 0J ' xC »'.» 



(19) 



where w 

Y.' ■» 



ij denotes the weight of trustworthy attribute A p 



■ 1. 



., = 1^- 

The trust value gives the comprehensive perception of cloud 
service trustworthiness, while the weights modify this trust value 
based on the relative importance of trustworthy attributes. 
Actually, a set of specified trustworthy attributes can be easily 
weighted by applying existing technologies such as those discussed 
in [32] . We omit the details for brevity. 

Decision support for personalized service 
selection. Multi-attribute trustworthiness evaluation is an im- 
portant step for making accurate service selection. We suppose 
that u m is the active customer, who requires trustworthy cloud 
service. While the evaluation results have arrived, a set of 
appropriate service candidates can be identified for u m by: 



CS"' = {cs„\trust„ > £„ 



■l,...,N}, 



(20) 



where trust,, denotes the trust value of denotes the selection 

threshold determined by a,„. We aim to remedy the shortcomings 
of evaluation-focused selection methods by avoiding the costly and 
time-consuming real-world service invocations. Note that when 
CS m = 0 the service selection for the active customer u m needs to 
be degrade by decreasing the parameter e m . 

Let £i = ... = £6 = 0.85, a set of trustworthy cloud services can be 
recommended for 



cs l 
cs 4 -- 



-- {CSI,CS4,CS$}, CS 2 

-- {cs 2 ,cs 4 ,cs 5 ,cs 6 }, CS 



{cs g ,cs 9 }, CS 3 = {cs u cs 9 ,cs ]0 } 
5 --{c Sl }, CS 6 = {c Sl ,cs s }, 



where the customer satisfaction for each attribute value is 
presented in Table 7. In practice, our approach makes it possible 
to deal with various types of trustworthiness-aware cloud service 
selections by combing the evaluation-focused and the prediction- 
focused methods. Note that if trustworthiness is not the only issue 
that affects customer's decision making, it is necessary to extend 
the selection process of our approach, e.g., price-oriented service 
filtering, into other attributes or indexes. 

Results 

In this section, abundance of experiments are conducted to 
show how to recommend trustworthy cloud service in the context 
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Figure 2. QoS value distributions, (a) and (b) depict the value 
distributions of response-time and throughput in our customer-service 
matrices, where "—1" indicates that the service invocation failed due to 
an http error. The ranges of response-time and throughput are 0-16.053 
seconds and 0-541.546 kbps, respectively. 
doi:1 0.1 371 /journal.pone.0097762.g002 

of large sparse assessment dataset, and to verify the efficiency of 
our CF approach. 



3 

z 
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Figure 4. Impact of preference and expectation, (a) and (b) depict 
the experimental results of preference parameter S and expectation 
hf* p , respectively. They indicates that S regulates the elimination rate of 
untrustworthy cloud services, whereas H exp controls the degree of 
customer's tolerance to untrustworthy service. 
doi:10.1371/journal.pone.0097762.g004 



Prototype implementation and results 

To demonstrate the effectiveness of the proposed service 
selection approach, we use Microsoft C# .NET to develop a 
prototype system. Based on literature [8,41,42], we find Availability 
and Performance are two commonly used trustworthy attributes. We 
utilize them to conduct trustworthiness-aware service selection, by 
including two types of historical records: response-time and throughput. 
Their evaluation styles and weights are summarized as Table 8. 




0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 

Threshold 

Figure 3. The number of recommended services for u 339 . Results 
are presented for the proposed cloud service selection approach, where 
the parameter £339 is varied from 0 to 1 in increment of 0.1. 
doi:1 0.1 371 /journal.pone.0097762.g003 



We employ an open QpS research dataset [43] to simulate the 
historical records of Performance and Availability in cloud service 
market. The QpS values for response-time and throughput were 
collected from 339 users over 5825 web services in a real-world 
environment. Since it is impractical to discover and distinguish all 
functionally-equivalent services at the selection time, we randomly 
select 100 services' QpS records, and construct two 339x100 
customer-service matrices for our experiment. Figure 2 shows the 
value distribution of response-time and throughput in user-service matrix. 
We cannot simply utilize these QoS records to analyze and rank 
the cloud services since these customer-service matrices are sparse 
assessment datasets, and cannot accurately interpret the trustwor- 
thiness status of all services. Suppose M339 is the active customer. 
The historical records made by K339 contains 9 and 7 missing 
values (on response-time and throughput, respectively) which will 
potentially affect his cloud service selection decision. Therefore, 
the proposed CF approach is employed to predict the missing 
attribute values. At this simulation experiment, the similarity 
parameter <x is set to 0.8 and remains so until the trust values for 
M339 are reported. Once the prototype system obtains the customer 
satisfactions by utilizing Eqs.(12)— (14), where the parameter (5 = 2 
and the expectation lT xp = 0.7, the active customer will receive the 
trust values of each service. We vary the selection parameter E339 
from 0 to 1 in increment of 0.1, and count the cloud services whose 
trust values surpass £339 (the number of recommended services for 
M339, i.e. I Cv 1 ). The experiment results are shown in Figure 3. 
Although we only study two trustworthy attributes in the 
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Table 8. Trustworthy attributes of cloud service. 





Attribute 


a j 


Evaluation style 


Weight 


?339 


?3 + 39 


Availability 


A, 


"cost" QoS value 


0.65 


0 


16.053 


Performance 


A 2 


"benefit" QoS value 


0.35 


0 


541.546 
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experiment, the proposed approach can be easily extended to 
other trustworthiness-aware service selection problems. When 
selecting the optimal trustworthy services from a set of function- 
ally-equivalent candidates, the entry data of our approach are the 
corresponding historical records (i.e., QoS values or customer 
ratings), the active customer's preference and expectation towards 
service trustworthiness, and the selection parameter. 

Impact of 8 and T exp 

Different customers have different preference and expectation 
in trustworthy service selections. Instead of risk-neutral attribute 
utility, we use the customer satisfaction C,„„ to identify the 
perceived trustworthiness of delivered services. To evaluate the 
impact of customer's preference and expectation, we have conduct 
additional experiments with variable parameters 5 and /f* 5 . In 
these experiments, we first vary 3 from 2 to 6 in increment of 2, 
where the expectation FT xp is fixed at 0.7 first. Later, we set S to 2, 



and vary IF xp from 0.7 to 0.9 in increment of 0.1. Figure 4 (a) 
shows the experimental results of preference parameter 5 and 
Figure 4 (b) shows the experimental results of expectation H rxp . 
The parameters 3 and //' xp jointly determine how to derive the 
customer satisfaction from attribute utility to approximate the 
active customer's attitude towards profit and risk. 

Performance comparison of CF approaches 

In this work, we present an enhanced item-based CF approach 
(i.e., JV-PCC) to predict the missing attribute values for cloud 
service selection. Our approach engages the significances {J nj and 
VD n J) to improve the accuracy of similarity estimation. To study 
the prediction performance, we compare JV-PCC with two 
existing item-based approaches: Item-based CF adopting PCC 
(IPCC) [44], and Extended PCC approach (f-PCC) [31]. 

Evaluation metric. We use Mean Absolute Error (MAE) and 
Root Mean Square Error (RMSE) to evaluate the prediction 



response-time 



response-time 




15 20 
k 

response-time 



15 20 

k 

throughput 



30 




Figure 5. Impact of neighborhood size k. (a) and (b) depict the MAE fractions of JV-PCC, f-PCC and IPCC for response-time and throughput, while 
(c) and (d) depict the RMSE fractions. It can be observed that JV-PCC achieves smaller MAE and RMSE consistently than f-PCC for both response-time 
and throughput. Regardless of JV-PCC or f-PCC, as k increases, MAE and RMSE drop at first, indicating that better performance can be achieved by 
employing more similar services' records to generate the predictions. However, when k surpasses a specific level (i.e. /t = 25), they fail to drop with a 
further increase in k, which were caused by the limited number of similar neighbors. 
doi:1 0.1 371 /journal.pone.0097762.g005 
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performance of our approach in comparison with other approach- 
es. MAE and RMSE are defined as: 



MAE-- 



RMSE = 



(22) 



where q mfl and q m<n are the predicted QpS value and the actual 
value, respectively. 

Experimental setup and results. The size of top-£ similar 
service set plays an important role in CF approach, which 
determines how many neighbors' historical records are employed 
to generate predictions. To study the impact of neighborhood size 
k, we separate the customer-service matrices into two parts: 
training set (80% historical records in the matrix) and test set (the 
remaining 20% records). We set the density to 50%, the 
significance parameter a to 0.7, and vary k from 5 to 30 in 
increment of 5. Figure 5 shows the experimental results for response- 
time and throughput. Under the same simulation condition, JV-PCC 
and f-PCC significandy outperform IPCC. The observations also 
suggest that better accuracy can be achieved by our model when 
more historical records are available in the service selection study. 

Conclusions 

Trustworthiness-aware service selection is a critical issue among 
cloud computing and service-oriented architecture communities. 



In this paper, we propose a personalized service selection 
approach which takes into account the missing value prediction 
and the multi-attribute evaluation requirements. We find that the 
proposed approach can tackle various types of trustworthiness- 
aware selection problems in cloud service market. Meanwhile, the 
experimental results demonstrate that the proposed CF approach 
significandy improves the prediction performance as compared 
with other competing item-based approaches. 

Employing untrustworthy cloud service will expose users to 
high-risk IT structure, resulting in a host of intra-organizational 
hazards that detriment the organization and disrupt the normal 
operations [45]. In the present work, we can only look into the 
static approach for trustworthy service selection, and we will 
investigate more types of trustworthiness evaluation models (e.g. 
probability model, dynamic model, etc) in the future since different 
cloud service applications may have different selection criteria and 
data structures. 
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